Not sure if the CI failures are legit, as I have tests passing locally...?

I've test your PR and seems something wrong with it. After plan I've got an error:

│ Error: kafka: client has run out of available brokers to talk to: 5 errors occurred:
│ 	* EOF
│ 	* EOF
│ 	* EOF
│ 	* EOF
│ 	* EOF

Same error in the pipeline tests https://github.com/Mongey/terraform-provider-kafka/actions/runs/7386689599/job/20093815910?pr=357

Without this patch all works good

@SanchosPancho I'd been running make testacc locally against the docker-compose'd brokers which was working.

When running testacc under docker, it does indeed fail, but seemingly from being unable to reach the brokers on localhost:9092. It seems like the env var for KAFKA_BOOTSTRAP_SERVERS wasn't correctly set or used in the Makefile, so the provider wasn't using the internal advertised listeners (kafka-{1..3}:9090), instead attempting to use the external localhost:9092, which wouldn't be reachable inside the testacc container without setting network: host...

I've made changes in be82e9b which should allow the testacc container to reach the brokers via the docker-compose network on 9090 (this would let the old CircleCI test approach work again).

I've just been running

docker compose -f docker-compose.yaml -f docker-compose.testacc.yaml up --abort-on-container-exit testacc

successfully on my end.

I think some of the tests are already racy anyway, but I've run this 5 times now and got 5 passes in a row. Could you try on your side?

Also sounds like some overlap with #383 which is nothing to do with this PR?

@eh-steve I've try it on real cluster, but I think that depends on kafka version, I've fail on old 2.3+ kafka cluster
seems provider 0.6 doesn't work correctly with old kafka instances

Yeah I guess if you wanted to test this PR you could locally cherry pick the first commit onto an older master.

I've also released it in the registry under eh-steve/kafka version 0.5.5 so you could test with that. The parent master commit for that release is 993494b, which may have already introduced the problem though?

Hey @eh-steve , Thanks for the above optimizations. Does the above reduce the time execution, on regular terraform plan commands for clusters that have tons of ACLs ?

BTW is there any help you need, to push forward with this PR?

Hey @eh-steve , Thanks for the above optimizations. Does the above reduce the time execution, on regular terraform plan commands for clusters that have tons of ACLs ?

Yeah it should

BTW is there any help you need, to push forward with this PR?

Just waiting for an approval and a CI run

Hi @Mongey, thanks for the run, I can’t reproduce these CI failures locally and one of them seems to be related to GPG? Is master CI passing?

hey @eh-steve thanks for the PR! looks great!
The build snapshots never really works for PR's due to GPG, I generally just ignore that failure.

I pulled the branch down locally and tested -- it seems to work for me.
Not sure what's going on with CI - it is quite flakey